{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "70996d8a",
   "metadata": {},
   "source": [
    "# IBM watsonx.ai\n",
    "\n",
    ">[WatsonxLLM](https://ibm.github.io/watsonx-ai-python-sdk/fm_extensions.html#langchain) is a wrapper for IBM [watsonx.ai](https://www.ibm.com/products/watsonx-ai) foundation models.\n",
    "\n",
    "This example shows how to communicate with `watsonx.ai` models using `LangChain`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea35b2b7",
   "metadata": {},
   "source": [
    "## Setting up\n",
    "\n",
    "Install the package `langchain-ibm`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "2f1fff4e",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install -qU langchain-ibm"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f406e092",
   "metadata": {},
   "source": [
    "This cell defines the WML credentials required to work with watsonx Foundation Model inferencing.\n",
    "\n",
    "**Action:** Provide the IBM Cloud user API key. For details, see\n",
    "[documentation](https://cloud.ibm.com/docs/account?topic=account-userapikey&interface=ui)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "11d572a1",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "from getpass import getpass\n",
    "\n",
    "watsonx_api_key = getpass()\n",
    "os.environ[\"WATSONX_APIKEY\"] = watsonx_api_key"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c59782a7",
   "metadata": {},
   "source": [
    "Additionaly you are able to pass additional secrets as an environment variable. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f98c573c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "os.environ[\"WATSONX_URL\"] = \"your service instance url\"\n",
    "os.environ[\"WATSONX_TOKEN\"] = \"your token for accessing the CPD cluster\"\n",
    "os.environ[\"WATSONX_PASSWORD\"] = \"your password for accessing the CPD cluster\"\n",
    "os.environ[\"WATSONX_USERNAME\"] = \"your username for accessing the CPD cluster\"\n",
    "os.environ[\"WATSONX_INSTANCE_ID\"] = \"your instance_id for accessing the CPD cluster\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e36acbef",
   "metadata": {},
   "source": [
    "## Load the model\n",
    "\n",
    "You might need to adjust model `parameters` for different models or tasks. For details, refer to [documentation](https://ibm.github.io/watsonx-ai-python-sdk/fm_model.html#metanames.GenTextParamsMetaNames)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "407cd500",
   "metadata": {},
   "outputs": [],
   "source": [
    "parameters = {\n",
    "    \"decoding_method\": \"sample\",\n",
    "    \"max_new_tokens\": 100,\n",
    "    \"min_new_tokens\": 1,\n",
    "    \"temperature\": 0.5,\n",
    "    \"top_k\": 50,\n",
    "    \"top_p\": 1,\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b586538",
   "metadata": {},
   "source": [
    "Initialize the `WatsonxLLM` class with previously set parameters.\n",
    "\n",
    "\n",
    "**Note**: \n",
    "\n",
    "- To provide context for the API call, you must add `project_id` or `space_id`. For more information see [documentation](https://www.ibm.com/docs/en/watsonx-as-a-service?topic=projects).\n",
    "- Depending on the region of your provisioned service instance, use one of the urls described [here](https://ibm.github.io/watsonx-ai-python-sdk/setup_cloud.html#authentication).\n",
    "\n",
    "In this example, we’ll use the `project_id` and Dallas url.\n",
    "\n",
    "\n",
    "You need to specify `model_id` that will be used for inferencing. All avaliable models you can find in [documentation](https://ibm.github.io/watsonx-ai-python-sdk/fm_model.html#ibm_watsonx_ai.foundation_models.utils.enums.ModelTypes)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "359898de",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_ibm import WatsonxLLM\n",
    "\n",
    "watsonx_llm = WatsonxLLM(\n",
    "    model_id=\"ibm/granite-13b-instruct-v2\",\n",
    "    url=\"https://us-south.ml.cloud.ibm.com\",\n",
    "    project_id=\"PASTE YOUR PROJECT_ID HERE\",\n",
    "    params=parameters,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2202f4e0",
   "metadata": {},
   "source": [
    "Alternatively you can use Cloud Pak for Data credentials. For details, see [documentation](https://ibm.github.io/watsonx-ai-python-sdk/setup_cpd.html).    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "243ecccb",
   "metadata": {},
   "outputs": [],
   "source": [
    "watsonx_llm = WatsonxLLM(\n",
    "    model_id=\"ibm/granite-13b-instruct-v2\",\n",
    "    url=\"PASTE YOUR URL HERE\",\n",
    "    username=\"PASTE YOUR USERNAME HERE\",\n",
    "    password=\"PASTE YOUR PASSWORD HERE\",\n",
    "    instance_id=\"openshift\",\n",
    "    version=\"4.8\",\n",
    "    project_id=\"PASTE YOUR PROJECT_ID HERE\",\n",
    "    params=parameters,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "96ed13d4",
   "metadata": {},
   "source": [
    "Instead of `model_id`, you can also pass the `deployment_id` of the previously tuned model. The entire model tuning workflow is described [here](https://ibm.github.io/watsonx-ai-python-sdk/pt_working_with_class_and_prompt_tuner.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "08e66c88",
   "metadata": {},
   "outputs": [],
   "source": [
    "watsonx_llm = WatsonxLLM(\n",
    "    deployment_id=\"PASTE YOUR DEPLOYMENT_ID HERE\",\n",
    "    url=\"https://us-south.ml.cloud.ibm.com\",\n",
    "    project_id=\"PASTE YOUR PROJECT_ID HERE\",\n",
    "    params=parameters,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c25ecbd1",
   "metadata": {},
   "source": [
    "## Create Chain\n",
    "Create `PromptTemplate` objects which will be responsible for creating a random question."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "c7d80c05",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.prompts import PromptTemplate\n",
    "\n",
    "template = \"Generate a random question about {topic}: Question: \"\n",
    "prompt = PromptTemplate.from_template(template)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79056d8e",
   "metadata": {},
   "source": [
    "Provide a topic and run the `LLMChain`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "dc076c56",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'topic': 'dog', 'text': 'Why do dogs howl?'}"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain.chains import LLMChain\n",
    "\n",
    "llm_chain = LLMChain(prompt=prompt, llm=watsonx_llm)\n",
    "llm_chain.invoke(\"dog\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f571001d",
   "metadata": {},
   "source": [
    "## Calling the Model Directly\n",
    "To obtain completions, you can call the model directly using a string prompt."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "beea2b5b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"Man's best friend is his dog. \""
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Calling a single prompt\n",
    "\n",
    "watsonx_llm.invoke(\"Who is man's best friend?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "8ab1a25a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LLMResult(generations=[[Generation(text='The fastest dog in the world is the greyhound, which can run up to 45 miles per hour. This is about the same speed as a human running down a track. Greyhounds are very fast because they have long legs, a streamlined body, and a strong tail. They can run this fast for short distances, but they can also run for long distances, like a marathon. ', generation_info={'finish_reason': 'eos_token'})], [Generation(text='The Beagle is a scent hound, meaning it is bred to hunt by following a trail of scents.', generation_info={'finish_reason': 'eos_token'})]], llm_output={'token_usage': {'generated_token_count': 106, 'input_token_count': 13}, 'model_id': 'ibm/granite-13b-instruct-v2', 'deployment_id': ''}, run=[RunInfo(run_id=UUID('52cb421d-b63f-4c5f-9b04-d4770c664725')), RunInfo(run_id=UUID('df2ea606-1622-4ed7-8d5d-8f6e068b71c4'))])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Calling multiple prompts\n",
    "\n",
    "watsonx_llm.generate(\n",
    "    [\n",
    "        \"The fastest dog in the world?\",\n",
    "        \"Describe your chosen dog breed\",\n",
    "    ]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d2c9da33",
   "metadata": {},
   "source": [
    "## Streaming the Model output \n",
    "\n",
    "You can stream the model output."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "3f63166a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "My favorite breed of dog is a Labrador Retriever. Labradors are my favorite because they are extremely smart, very friendly, and love to be with people. They are also very playful and love to run around and have a lot of energy. "
     ]
    }
   ],
   "source": [
    "for chunk in watsonx_llm.stream(\n",
    "    \"Describe your favorite breed of dog and why it is your favorite.\"\n",
    "):\n",
    "    print(chunk, end=\"\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
